A conventional arithmetic processing device, which realizes a convolutional neural network including a plurality of process layers, includes a storage device for each process layer to store the entire output of the process layer, to perform every process of the process layer and store the output of every process in the storage device, and perform the process of the succeeding layer using the stored numerical values. The storage device has at least one arrangement of numerical values arranged in a row direction and a column direction. When there is a plurality of arrangements, the arrangements are arranged in a direction intersecting with a plane including a row direction and a column direction. The direction with which the plane intersects is referred to as a depth direction.
As a process to be performed by a process layer, there is, for example, a convolution process which is a process of obtaining the sum of products of corresponding numerical values between an arrangement of specific numerical values, which is referred to as a kernel, and an arrangement of numerical values having the same depth as the kernel. Each numerical value of the kernel is referred to as a weight. The process of obtaining the sum of products may be followed by a process of adding a specific numerical value, which is referred to as a bias, to the value of the sum of products.
An activation function process may further be performed to the result of the process of obtaining the sum of products or the result of the process of adding the bias. The activation function process is defined as a process of assigning a specific function value to a specific numerical value, for example, assigning a value of tan h (x) to a numerical value x. Another example is a Rectified Linear Unit process of assigning the value x or zero, either one of a larger one, to the numerical value x.
The convolution process may be followed by a pooling process which is a process of appointing one numerical value to represent a group of specific numerical values. As a method of this process, there are a method of extracting a maximum value of the specific numerical values, a method of calculating an arithmetic mean of the specific numerical values, etc. The depth of a kernel in the convolution process is generally the depth of the arrangement of numerical values to be subjected to the convolution process. The depth of a group of specific numerical values in the pooling process is 1 in general.
As described above, the conventional arithmetic processing device has a storage device for each process layer to store the entire output of the process layer, to perform every process of the process layer and store the output of every process in the storage device, and perform the process of the succeeding layer using the stored numerical values. Therefore, it is required to have a storage device for each process layer to store the entire output of the process layer, which requires a large circuit area, and as a result, causes increase in production cost.